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Abstract: 

This document describes a method to demonstrate that a UAS, operating in the NAS, can avoid collisions 
with an equivalent level of safety compared to a manned aircraft. The method is based on the calculation 
of a collision probability for a UAS , the calculation of a collision probability for a base line manned 
aircraft, and the calculation of a risk ratio given by: 

Risk Ratio = P (collision _UAS)/P (collision jnanned). 

A UAS will achieve an equivalent level of safety for collision risk if the Risk Ratio is less than or equal to 
one. Calculation of the probability of collision for UAS and manned aircraft is accomplished through 
event/fault trees. 


Status: 


TBD 


Limitations on use: 

Some of the probability values assigned to the nodes in the event trees are estimations and must 
be validated based on analysis, simulation and/or flight experiments. The risk ratio is calculated 
for the baseline of a manned aircraft without collision avoidance on-board. The collision 
probability calculations are for the own ship, manned and unmanned, to avoid the traffic 
aircraft. It does not take into account the capability of the traffic aircraft to avoid the own ship. 
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Executive Summary 

The Access 5 program objective is to develop methods and guidelines to routinely operate 
unmanned aerial vehicles in the National Airspace System (NAS), similar to manned aircraft 
flight. To achieve the objective of routine operations of Unmanned Aircraft System (UAS) in the 
NAS, UASs must integrate safely into the NAS while causing the minimum impact on the 
existing system. 

Operation of a UAS in the NAS requires that the aircraft perform a set of functions which will 
meet mission objectives. A critical element of these functions is to sense and avoid other traffic 
in the airspace. Manned aircraft operating in the NAS have a low risk of mid air collision due to 
a multilayer approach to collision avoidance. These layers consist of operational procedures, Air 
Traffic Control services, on-board collision avoidance systems, and see and avoid. 

This document describes a method to demonstrate that a UAS, operating in the NAS, can avoid 
collisions with an equivalent level of safety compared to a manned aircraft. The methods is based 
on the calculation of a collision probability for a UAS, the calculation of a collision probability 
for a baseline manned aircraft, and the calculation of a risk ratio given by: 

Risk Ratio = P(collision_UAS)/P(collision_manned) 

A UAS will achieve an equivalent level of safety for collision risk if the risk ratio is less than or 
equal to one. 

Calculation of the probabilities of collision for UAS and manned aircraft are accomplished 
through an event/fault tree. Node probabilities of the event/fault trees are based on historical 
data, previous safety studies, operational data, discussions with pilots and aviation experts, 
simulation experiments, flight tests, and analysis. Event trees are included in this document 
representing some candidate sense and avoid configurations. 
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1. Introduction 


1.1 Purpose 

The purpose of this report is to describe a safety analysis method to estimate the relative risk of 
collision of an Unmanned Aircraft System (UAS) in the National Airspace System (NAS), as 
compared to a manned aircraft. Safety and risk ratios are based on the calculation of the 
probability of a Mid-Air Collision (MAC). The safety analysis also evaluates the sensitivity of 
the factors contributing to the collision probability. An event/fault tree is used to calculate MAC 
probabilities. Two representative event trees are included in this report. 

1.2 Scope 

The safety analysis in this document addresses risk of collision of UAS s in the NAS operating in 
class A airspace. It considers the probability of collision under non failure conditions and under 
some failure conditions that directly affect the collision avoidance functionality. The collision 
probability calculation does not consider failure conditions such as engine failure, hydraulic 
system failure, structural failure, etc. These failure conditions and the hazards associated with 
them will be addressed elsewhere by the Access 5 Reliability Work package. 

Probability of MAC is estimated using event/fault trees. The probabilities of occurrence of 
primary events will be obtained, calculated, or estimated using historical data, previous safety 
studies, existing operational data, analytical methods, simulation and flight test. The event trees 
include physical and operational components. The example event trees included in this report 
estimate the probability of MAC for UASs using Mode-S/directional antenna surveillance and 
GPS/ADS-B surveillance with Access 5 developed collision avoidance logic. It must be noted, 
however, that the Access 5 project is not proposing or endorsing the use of Mode-S/directional 
antenna and/or GPS/ADS-B as the solution system for UAS sense and avoid. 


1.3 Objective 

Estimation of MAC probabilities for various collision avoidance and surveillance systems will 
complement the definition of Equivalent Level of Safety (ELOS) defined in “ELOS Comparable 
to See-and- Avoid Requirements for Manned Aircraft” [8], It will provide a quantified evaluation 
of collision avoidance and surveillance systems. It will be used to support the development of 
functional requirements and reliability requirements. The MAC probability estimation is 
performed in the context of an operational environment and using assumptions from the 
proposed concept of operations. A MAC, as defined in section 1.5, provides a geometrical 
characterization to be used in simulation and analytical studies. A MAC has been defined such 
that any encounter of this nature between aircraft is considered a catastrophic event. No attempt 
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is made to characterize the geometry or contact points of a collision, possibility of surviving a 
collision, or other lower level details. 

1.4 Project Sponsors and Participation 

The Access 5 project is sponsored by the National Aeronautics and Space Administration 
(NASA). The Access 5 Project Office is located at NASA Dryden Flight research Center, 
Edwards, California. This report contains work performed under the Collision Avoidance Work 
Package 2, Safety Analysis Task. 

1.5 Definitions 

• Mid-Air Collision (MAC) - An encounter between aircraft that results in a separation from 
center of gravity to center of gravity of 200 feet or less horizontally and 65 feet or less 
vertically. See Section 2.5.1. 

• Critical Near Mid-Air Collision (NMAC) is defined as separation distance center of gravity 
to center of gravity of 500 feet or less horizontally and 100 feet or less vertically [4, 9]. A 
Near Mid-Air Collision has also been defined in references [16, 17] as “proximity of less 
than 500 feet to another aircraft” which is interpreted as the closest distance between aircraft 
structures. 

• Proximity Trajectory - A trajectory in which two aircraft will have a separation of one half 
(Vi) nautical mile or less horizontally and 500 feet or less vertically at the closest point of 
approach. This trajectory is defined as an encounter in which there is a possibility of a pilot 
identifying the traffic aircraft as a collision threat and taking evasive action which might lead 
to an induced collision. The closest point of approach distance is deliberately selected as a 
large distance to include any induced collision possibility. It is assumed that trajectories with 
a larger separation at the closest point of approach do not represent a credible threat for an 
induced collision. 

• RA encounter - An encounter between aircraft that would result in a resolution advisory 
from an operational TCAS II in one or both aircraft. Since the sensitivity of TCAS to issue 
RAs is selectable, an RA encounter cannot be directly associated with an encounter 
geometry. In many cases, an encounter in which an RA is issued will result in a NMAC if no 
changes in trajectories are performed. However, there are cases in which an RA is issued and 
no NMAC will result even if no evasive action is taken. For the purposes of analysis, an 
encounter between aircraft that does not result in an RA because the TCAS is not operating 
or installed, is still consider an RA encounter if an operational TCAS would have issued an 
RA. 
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• Separation - The standard en-route distances that two aircraft must maintain as dictated by 
the FAA. Typical distances are 5 Nautical Miles horizontally and 2000 feet vertically; 1000 
feet vertically for Reduced Vertical Separation Minima (RVSM). No separation standard 
distance applies to VFR flight. 

• Loss of separation - Current violation of separation minima (time = 0). 

• Conflict - A predicted loss of separation. The trajectories of the aircraft are such that a loss 
of separation will occur in the future (time > 0). 


2. Mid-Air Collision Probability Estimation 

The estimation of the probability of MAC is based on an event/fault tree analysis. The estimation 
is based, as far as possible, on a generic remotely operated aircraft, making minimum 
assumptions on the system implementation. The generic concept of operations and list of 
assumptions are included in this section. Each of the primary event probabilities will be 
calculated or assigned a value and the rationale for the values will be discussed. 

2.1 Risk Ratio Method 

One of the objectives of the Access 5 project is to determine the capabilities and requirements of 
UASs to assure that the risk of collision is no higher than for manned aircraft. The overall 
collision probability of an aircraft in the NAS depends on several factors including operational 
procedures, type and density of airspace, type and size of aircraft, air traffic control services, 
weather, etc. Figure 2.1 depicts the predominant layers in the NAS to prevent mid air collisions. 
Note that not all manned aircraft are required to have on-board Collision Avoidance Systems 
(CAS) and that implementations of UASs might or might not have CAS logic at the control 
station. 
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Figure 2.1. Primary Layers in the NAS to Prevent Mid Air Collisions. 

The method used to assess the collision avoidance capability of a remotely operated aircraft is to 
calculate a risk ratio. The risk ratio is the probability of collision per encounter of a UAS divided 
by the probability of collision per encounter of a manned aircraft. The risk ratio calculations in 
this report, Section 5, are for manned aircraft without collision avoidance equipment on-board. 
However, the event trees and the analysis structure include nodes and events which can be used 
to calculate a risk ratio including collision avoidance equipment on-board the manned aircraft. 

For the purpose of this safety study, the assumption is made that such factors as procedures and 
ATC services will contribute equally to the prevention of a collision for manned aircraft than for 
a UAS. Therefore, the risk ratio is based on the comparison of the see- and- avoid factor for a 
manned aircraft and the sense-and-avoid factor for UAS. 

The assumption that procedures and ATC services contribute equally to collision avoidance for 
UAS and manned aircraft is not necessarily a sound assumption. This will be addressed by 
further studies and simulations. Specifically, the simulation work package is developing 
comprehensive simulations which model the interaction of UAS, ATC, and other NAS users and 
operators. 

2.2 Concept of Operation 

A generic concept of operation is used to identify which parts of the system can have an impact 
on the collision avoidance functionality of the UAS. The operation of the UAS is depicted in 
Figure 2.2. 
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Figure 2.2. Functional Representation of UAS Operation. 

The aircraft is controlled from a ground/airborne Command and Control Center. A command and 
control (CC) link sends commands to the aircraft and receives status data from the aircraft 
including surveillance, systems status, location, flight parameters, and other. Voice 
communication between the UAS, ATC and traffic aircraft is relayed to the command and 
control center via the CC link, a separate link, or both. The CC link and the voice relay link 
could be implemented by a single link or by redundant and possibly dissimilar communication 
systems. 

The UAS is equipped with a Mode-S transponder. The manned aircraft, the UAS own ship and 
the UAS traffic might or might not have ADS-B capabilities. Depending on the implementation, 
the surveillance of the UAS own ship could be a variety of technologies including radar, infrared, 
optical in the visual spectrum, TCAS surveillance, ADS-B reception, TIS-B reception, or a 
combination of these or other technologies. The safety analysis can be used to determine the 
impact of these technologies on the overall collision risk and evaluate the alternatives. 
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2.3 Assumptions 

The assumptions listed in this section are for the purpose estimating basic event probabilities. 
They do not reflect the operational guidelines and procedures that could result from the Access 5 
project or other previous or current work. 


• The UAS is assumed to operate in the NAS without any special operational procedure 
distinct from manned aircraft. No segregation between UAS and manned aircraft is 
performed. 

• The UAS aircraft is Mode-S equipped. The Mode-S transponder might or might not have 
extended broadcast capability with ADS -B data. 

• All traffic in the UAS operational environment are Mode-S equipped. Traffic aircraft might 
or might not have Mode-S transponder with extended ADS-B broadcast capabilities. 


2.4 Event /Fault Tree 

The event/fault tree is composed of a set of basic (primary) events, probabilistic association of 
events represented by logic functions, and compound events that result from basic events or other 
compound events. The top-level event gives the probability of a Mid-Air Collision between two 
aircraft given that these aircraft are in a collision course or proximity course and does not take 
into account the collision avoidance capability of the traffic aircraft. That is, the collision 
avoidance calculation is only based on the capabilities of the ownship. The probabilistic gates 
used in the event tree are: 

• AND - An event occurs if and only if all its sub-events occur. The probabilistic AND 
calculation is the product, PI x P2, of the input events. 


• OR - An event occurs if any or all of its sub-events occur. The probabilistic OR calculation is 
the sum minus the product, PI + P2 - PlxP2, of the input events. In the case of a 3 input OR, 
the probability is given by: 

PI + (P2 + P3 - (P2xP3)) - Plx(P2 + P3 - (P2xP3)) 

distributing and rearranging, 

PI + P2 + P3 - PlxP2 - PlxP3 - P2xP3 + PlxP2xP3 


• XOR - (exclusive or) An event occurs if any of its sub events occur. The sub-events are 
mutually exclusive; that is, they cannot occur simultaneously. The probabilistic XOR 
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calculation is the sum, PI + P2, of the input events. An example of mutually exclusive events: 
PI, On MAC trajectory, P2, Not on MAC trajectory. 


2.4.1 Top Level Tree 


The top level tree, shown in Figure 2.3, is common to the manned aircraft see-and-avoid and to 
the UAS sense-and-avoid function. 
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Figure 2.3. Top Level Tree Common to Manned and Unmanned Aircraft. 

The probability of a Proximity trajectory when in an en-route environment is the probability that 
procedures in the NAS and air traffic control services have failed to maintain the required 
separation between aircraft. To calculate a risk ratio between manned and unmanned aircraft, this 
probability is set to 1. See Section 2.5.3 for a detailed discussion. 

The events On MAC trajectory and Not on MAC trajectory are the geometric probabilities that 
given a Proximity trajectory, the trajectory is a MAC trajectory or not. These nodes are mutually 
exclusive and their values are calculated in Section 2.5.1. Note that a MAC trajectory is a subset 
of a NMAC trajectory and a NMAC trajectory is a subset of a Proximity trajectory. The event No 
Maneuver/Unresolved represents the probability that the see/sense and avoid technique will fail 
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to avoid a collision. The event Maneuver Causes MAC represents the probability that the action 
taken to avoid a collision actually causes the collision where no collision would have occurred 
had no action been taken. The subtrees which lead to the compound events No 
Maneuver/Unresolved and Maneuver Causes MAC are specific to the avoidance technique and 
will be shown in Section 2.4.2 for manned see and avoid and Section 2.4.3 for unmanned sense 
and avoid. 

2.4.2 Baseline Event Subtrees for Manned Aircraft See and Avoid 

The baseline event subtrees estimate the probability that a manned aircraft will not avoid a 
collision (Subtree A, Figure 2.4) and that the avoidance actions by the pilot cause the collision 
when the aircraft were not in a collision trajectory (Subtree B, Figure 2.5). The probability values 
for the basic nodes are calculated in the Basic Event section, Section 2.5. The basic nodes are 
numerically labeled above the node box to match with the probability calculations. 
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Figure 2.4. Subtree A for Manned Aircraft. 
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Figure 2.5. Subtree B for Manned Aircraft. 


2.4.3 Event Subtrees for Unmanned Aircraft Sense and Avoid 

The event subtrees estimate the probability that an unmanned aircraft will not avoid a collision 
(Subtree A, Figure 2.6) and that the avoidance actions by the pilot cause the collision when the 
aircraft were not in a collision trajectory (Subtree B, Figure 2.7). 
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Figure 2.6. Subtree A for Unmanned Aircraft. 
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Figure 2.7. Subtree B for Unmanned Aircraft. 


2.5 Basic Events 

The probability of basic events will be obtained from different sources or calculated. In some 
instances, the probability will be obtained from simulation and experiments of other Access 5 
work package teams. The probability of failure of the command and control link (CC link) is 
such an example. 

The events are classified into geometry, equipment, human factors, and logic events. 
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2.5.1 Geometry Events 

Geometry events comprise events that mostly result from the physics of the encounter. 
Inevitably, there will be some secondary effects. For example, the probability of visual 
acquisition is directly affected by the geometry of the encounter. However, since the primary 
factor for visual acquisition is the pilot, this event is grouped under the human factor events. 


MAC and Not MAC probabilities, Nodes 1 and 2 

A mid air collision is defined as a separation when the center of gravities (eg) of two aircraft are 
200 feet or less of each other horizontally and 65 feet or less vertically. These distances are a 
conservative estimate of the volume required to clear most aircraft without structural contact. 
Very large aircraft such as the Boeing B747-400 (211 feet wingspan), Airbus A380-800 (262 feet 
wingspan), and Lockheed C5A Galaxy (223 feet wingspan) will require larger volumes. 
However, for simplification of simulation and analysis, the collision distance is set to the values 
indicated above. 

A critical near mid air collision is defined as an encounter where two aircraft are center of gravity 
to center of gravity 500 feet or less horizontally and 100 feet or less vertically [4, 9]. See Figure 
2.8. A proximity encounter is defined as a distance where two aircraft are one half (V 2 ) nautical 
mile or less horizontally and 500 feet or less vertically. 



1/2 NM 


Figure 2.8. Horizontal and Vertical view of MAC, NMAC and Proximity Volumes. 
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The protected zones for MAC, NMAC and Proximity have been defined such that the diameter 
(rather than the radius) are 200 feet, 500 feet and Vi NM, respectively. Therefore, an intrusion to 
these zones is when the protected zones of each aircraft overlap, Figure 2.9. 

< 200 feet 


MAC 



NMAC 




Figure 2.9. Intrusion into Protected Zone for MAC, NMAC, and Proximity. 

The probability that two aircraft are in a MAC trajectory given that they are in a Proximity 
trajectory is based on the definitions above. There are two cases to consider when calculating the 
probability of collision trajectories. The first is when both aircraft are on level flight. The second 
is when one or both aircraft are climbing or descending. 

The following examples demonstrate how the probabilities are calculated. In Figure 2.10, two 
aircraft are in a 180 degrees encounter (head on). The location of aircraft 1 along the y axis is 
assumed to have a uniform distribution. The location of aircraft 1 along the x axis does not 
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affect the closest point of approach distance. The location of aircraft 1 along the z axis (vertical) 
is also assumed to have a uniform distribution. 



Figure 2.10. Probability of MAC and Proximity Trajectories, Head-on Encounter. 

The probability of a collision trajectory given a Proximity trajectory (at level flight), P(ctlpt- 
level), is the probability that horizontally they are within 200 feet eg to eg, P(cth), times the 
probability that they are vertically within 65 feet, P(ctv). 

P(cth) = 200/3038 = 0.06583 

P(ctv) = 65/500 = 0.1300 

P(ctlpt-level) = P(ch) x P(cv) = 0.06583 x 0.1300 = 0.008558 
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Note that this probability is independent of the encounter horizontal angle (crossing angle). 
Figure 2.1 1 shows two aircraft with a 135 degrees encounter angle. Without loss of generality, let 
aircraft 1 and aircraft 2 have vertical speeds equal to 250 knots. Their vectorial speeds are: 

vxl = 250*cos(45) = 176.8 knots 
vyl = 250*cos(45) = 176.8 knots 
vx2 = -250 knots 
vy2 = 0 knots 

The relative speeds and angle of motion of aircraft 1 with respect to aircraft 2 are: 

vxr = 426.8 knots 
vyr = 176.8 knots 
Theta_r = 67.5 degrees 
y 

* 




Figure 2.11. Probability of MAC and NMAC Trajectories, 135° Encounter, Showing 

Closing Relative Horizontal Speed. 

A uniform distribution of aircraft 1 over the x and y axes will produce a probability P(cth) of 
horizontal collision trajectory equal to 0.06583 as before. The probability of vertical collision 
trajectory is unaffected by the horizontal angle. In general, the relative trajectory of aircraft 1 will 
tangentially intercept a circle around aircraft 2. The probability of horizontal MAC trajectory 
given a Proximity trajectory is the ratio of the MAC radius to the Proximity radius. 

For encounters where one or both aircraft are climbing or descending, the probability of a 
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collision trajectory given a proximity trajectory is based on the horizontal areas of MAC and 
Proximity, the vertical area and the relative angle of the aircraft as shown in Figure 2.12. The 
location of aircraft 1 is assumed to have uniform distributions on the vertical and horizontal axes. 



Figure 2.12. Probability of MAC Given Proximity, Climb/Descent. 

The horizontal areas of the MAC and Proximity volumes are: 

Area_h_MAc = tt 100 2 sq feet 
Area_h_proc = n 1 5 1 9 2 sq feet 

The vertical area of the MAC and Proximity volumes are: 

Area_v_MAC = 65 x 200 sq feet 
Area_v_proc = 500 x 3038 sq feet 

The probability of MAC trajectory given a Proximity trajectory is: 

P(ctlnmt-non-level) = Area M Ac/AreapRoc 

= (Area_h_MAC+ Area_v_MAc/Tan a)/(Area_h_proc + Area_v_proc/Tan a) 

For an aircraft at 250 knots and 500 feet per minute climb, the trajectory angle with respect to the 
horizon is only 1.13 degrees. This give a probability of MAC trajectory given a Proximity 
trajectory of: 

P(ctlnmt-non-level) = Area M Ac/AreaNMAc = 0.008461 
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The encounter probabilities of Section 3 are used to obtain the probability of level flight 
encounters and non-level flight encounters. Nineteen encounter classes are presented in Section 3 
which were obtained from [4], For each of these encounters, it is determined if the encounter is a 
level or non-level flight encounter. Classes 2, 3a, 3b, 7, 8, 9, 10, 12, 13, 17, 18, and 19 are level 
flight encounters at the closest point of approach. Classes la, lb, 4, 5, 6, 11, 14a, and 14b, 15, 

16, are non-level flight encounters at the closest point of approach. The probability of a level 
flight encounter is: 

P(level) = P(class 2) + P(class 3a) + P(class 3b) + P(class 7) + P(class 8) + 

P(class 9) + P(class 10) + P(class 11) + P(class 12) + P(class 13) + 

P(class 17) + P(class 18) + P(class 19) 

= 0.27162 

The probability of a non-level flight encounter is: 

P(non-level) = P(class la) + P(class lb) + P(class 4) + P(class 5) + P(class 6) + 

P(class 11) + P(class 14a) + P(class 14b) 

= 0.72837 

The probability of a collision trajectory given a proximity trajectory is computed taking into 
account the probability of encounters: 

P(ctlpt) = P(ctlpt-level) x P(level) + P(ctlpt-non-level) x P(non-level) 

= 0.008558 x 0.27162 + 0.008461 x 0.72837 
= 0.002325 + 0.006163 
= 0.008488 

Node 1 = 0.008488 
Node 2 = 0.991512 

2.5.2 Equipment Events 

These events result from the behavior of the equipment which is installed on the aircraft for the 
purpose of collision avoidance. 

Sensor does not Detect, Node 28 (unmanned) 

Probability that the detection sensor fails to acquire the target traffic. The sensor could be radar, 
infrared, visual spectrum, transponder query (similar to Mode-S used in TCAS), ADS-B receiver, 
TIS receiver, other technology, or a combination of sensors. The probability that a Mode-S 
sensor fails to acquire a target is obtained from reference [1]. The probability that an ADS-B 


19 



receiver does not detect an ADS-B equipped aircraft is calculated by the tree in Figure 2.13. 


Sensor does not 
detect 


Traffic aircraft Reception failure Own aircraft Erroneous ADS-B 

ADS-B equipment ADS-B equipment DATA 

failure Failure 


Figure 2.13. Probability of Not Detecting Traffic Aircraft. 

Equipment failure probability is based on guidelines of the RTCA DO-242A document [5]. 
Section 3. 3. 6.2, Failure Mode and Availability Considerations , specify that: 

Where the ADS-B System is used as a supplemental means of surveillance, the ADS- 
B system is expected to be available with a probability of at least 0.95 for all 
operations, independent of the availability of appropriate inputs to the ADS-B 
system. Where the ADS-B System is used as a primary means of surveillance, the 
system is expected to be available with a probability of at least 0.999 for all air-air 
operations. 

Under the assumptions made in this analysis, the unmanned aircraft is operated under ATC 
services in Class A airspace. The primary means of surveillance is Primary System Radar and 
Secondary System Radar, PSR/SSR. However, for this analysis, ADS-B is also used for the 
additional functionality of sense and avoid. It is therefore expected that when used for sense and 
avoid in addition to supplemental surveillance, a higher availability requirement will be imposed. 
A value of between 0.99 and 0.999 is assigned to the equipment failure probability. 

The probability of reception failure is calculated based on the work of Ronald Staab, reference 
[12]. Assuming a 20 nautical mile range and 5 FRUIT (False Returns Unsynchronized in Time) 
the probability of receiving a single message from the traffic aircraft is approximately 0.985. For 
a 1 second rate of transmission and a 5 second interval, the probability of missing all messages in 
the 5 second interval is: 

Reception Failure = (1 - 0.985) 5 = 7.59 x 10" 10 
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An ADS-B receiver which loses a traffic aircraft messages for longer than 5 seconds will render 
the accuracy and integrity of that aircraft as invalid, as defined in RTCA DO-260 and DO-260A 
[14,15]. It should be noted that missing all messages from the traffic for a 5 second interval will 
not prevent an aircraft from reacquiring the traffic. If the assumption is made that missing to 
detect a traffic aircraft for more than 5 seconds will significantly compromise the ability to sense 
and avoid, then this reception failure probability could be used. 

An aircraft reporting a location through an ADS-B transmission is expected to be no farther away 
than a containment radius, Figure 2.14. 


Integrity Containment 

. • " " ■ f' 

Reported location of aircraft 

4 


Figure 2.14. Integrity Containment Radius for ADS-B Reported Location. 

The Surveillance Integrity Level parameter, SIL, defines how often an aircraft will be outside of 
this containment radius as measured from the reported location. The values for this parameter are 
shown in Table 2.1. 


SIL 

Probability of Exceeding Containment Radius Reported in NIC without 

Being Detected 

0 

Unknown 

1 

1 x 10' 3 per flight hour or per operation 

2 

1 x 10“ 5 per flight hour or per operation 

3 

1 x 10 per flight hour or per operation 


Table 2.1. Surveillance Integrity Level, SIL, and Meaning. 

Depending on the requirements imposed on the sense and avoid equipment, the probability of 
undetected erroneous data will be in the range of 1 x 10" 3 to 1 x 10" 7 with a corresponding SIL 
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value of 1 to 3. An important factor that is undefined by SIL is the latency of the undetected 
erroneous data. For example, a SIL of 1 represents that no more than one hour of undetected 
erroneous data will be transmitted per 1000 flight hours (or operations). However, a continuous 
one hour of undetected erroneous data followed by 999 hours of good data is much more severe 
than 1 second of undetected erroneous data followed by 999 seconds of good data. The worst 
case is assumed in this analysis, where an aircraft could transmit erroneous data exceeding its 
containment radius for long periods of time but that the ratio of erroneous data time to good data 

time is in the range 1 x 10" 3 to 1 x 10" 7 . 


Node 28 = 

P(Mode-S) = 5.000 x 10" 3 from reference [1] 

P(ADS-B) = 1 . 100 x 10“ 2 to 1 .000 x 10 3 

Data Down-Link Fails, Node 29 (unmanned) 

The probability that, at the time of the encounter, the data link which sends traffic information 
from the unmanned aircraft to the control station and the traffic display has failed. The 
probability is calculated as the amount of flight time with a failed link per flight hour. This 
probability is affected by the reliability of the equipment and by the operational rules. Given a 
down-link failure, the aircraft might be allowed to continue in normal flight or an abnormal flight 
termination might be executed. The probability for this node is based on the hazard analysis 
conducted by the Reliability Work Package of Access 5. The Hazard Analysis classifies the loss 
of down link as a minor hazard when an auto-land function is present on the unmanned aircraft 
and as major hazard when no auto-land function is present. From Advisory Circular AC/AMJ 
25.1309 the level of hazard is related to failure probability as shown in Table 2.2. 
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Effect on 

Aircraft 

No effect on 
operational 
capability of 

safety 

Slight 

reduction in 
functional 
capabilities or 

safety 

margins 

Significant 
reduction in 
functional 
capabilities or 

safety 

margins 

Large 

reduction in 
functional 
capabilities or 

safety 

margins 

Normally with 
hull loss 

Average 
Probability per 
flight hour 

No 

requirement 

< 10' 3 

< 10' 5 

< 10' 7 

< 10' 9 

Classification 
of Failure 
Conditions 

No safety 
Effect 

Minor 

Major 

Hazardous 

Catastrophic 


Table 2.2. Hazard Severity and Probability of Occurrence. 

Based on a minor and major classification, the probability of a down link failure is given the 
value in the range of 10" 3 to 10" 5 . 

Node 29 = 1.0 x 10" 3 to 1.0 x 10" 5 


MFD/ Driver Fails, Node 30 

Probability that the traffic data is available at the control station but it is not displayed due to 
system display failure. This number is based on a single display with a 15 to 40 thousand hours 
mean time to failure for traffic displays. 

Node 30 = 6.666 x 10‘ 5 to 2.500 x 10' 5 

2.5.3 Human Factors 

These events are the result of the behavior of the pilots and the air traffic controller, if the ATC is 
involved in the event. 

Pilot(s) do not see traffic, Node 3 (manned) 

This node is the probability that a pilot or pilots will not see the traffic aircraft. This probability 
is calculated using a 3 dimensional visual acquisition program described in Section 4. 

A pilot might see the traffic aircraft a few seconds before collision. For the purpose of estimating 
the outcome of the encounter, seeing the aircraft a few seconds before the encounter is equivalent 
to not seeing it at all as their will not be sufficient time to react. The threshold for calculating 
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whether or not the pilot sees the traffic before collision is set to 5 seconds. 

Meteorological conditions affect the capacity of a piloted aircraft to see an intruder aircraft. 
Meteorological conditions will also affect sensing devices such as Lidar, and video surveillance. 
Visibility and weather conditions are factored in the probability of visual acquisition. Section 3 
defines the encounter geometries used to calculate visual acquisition probabilities. Encounter 
geometries are grouped into Classes and the crossing angles are varied. Table 2.3 gives the 
probability of the own ship aircraft not seeing the target aircraft as a function of crossing angle 
for encounter Class la. Table 2.4 gives the average probability over the crossing angle range for 
the 19 class encounters presented is Section 3. The probabilities in the table are calculated for 
the following parameters: 

traffic aircraft: 757-200 climb/descent rates: 500 feet/min 

true air speed, own: 250 knots visual range: 20 nautical miles 

true air speed, traffic: 460 knots tau 2: time to collision: 5 seconds 

number of pilots, own: 2 


Crossing 

Angle, 

degrees 

Probability 
No Visual 

Crossing 

Angle, 

degrees 

Probability 
No Visual 

Crossing 

Angle, 

degrees 

Probability 
No Visual 

0 

1.0000 

65 

0.0000 

130 

0.0744 

5 

1.0000 

70 

0.0000 

135 

0.1000 

10 

1.0000 

75 

0.0001 

140 

0.1313 

15 

1.0000 

80 

0.0003 

145 

0.1690 

20 

1.0000 

85 

0.0007 

150 

0.2142 

25 

1.0000 

90 

0.0016 

155 

0.2666 

30 

1.0000 

95 

0.0032 

160 

0.2897 

35 

0.0000 

100 

0.0059 

165 

0.3129 

40 

0.0000 

105 

0.0102 

170 

0.3363 

45 

0.0000 

110 

0.0166 

175 

0.3600 

50 

0.0000 

115 

0.0256 

180 

0.3839 

55 

0.0000 

120 

0.0378 



60 

0.0000 

125 

0.0539 




Table 2.3. Probability of No Visual Acquisition as Function of Crossing Angle. 
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Crossing angles between 0 and 30 degrees give a probability of one because the ownship is 
unable to see the intruder aircraft given the field of regard from the cockpit and the geometry of 
the encounter. For angles between 35 and 70 degrees, the ownship is able to see the intruder out 
of the window and due to the geometry, the closing speeds are relatively slow, given ample 
opportunity for a pilot who is scanning the horizon to see the intruder. As the encounter becomes 
more a head-on encounter, for angles 75 to 180 degrees, the closing speeds become greater, 
making it more difficult to see the aircraft tau seconds before collision. 


Class 

Average 

Class 

Average 

Class 

Average 


Probability 


Probability 


Probability 


No visual 


No visual 


No visual 

la 

0.2640 

6 

0.2606 

13 

0.2610 

lb 

0.2612 

7 

0.2661 

14a 

0.2619 

2 

0.2661 

8 

0.2610 

14b 

0.2619 

3a 

0.2610 

9 

0.2597 

15 

0.2626 

3b 

0.2610 

10 

0.2661 

16 

0.2624 

4 

0.2606 

11 

0.2626 

17 

0.2661 

5 

0.2626 

12 

0.2661 

18 

0.2610 


19 

0.2597 


Table 2.4. Average Probability of Ownship not Seeing Traffic. 

Based on a weighed average, taking into account the frequency of the encounters (see Table 3.1), 
the probability is set to: 

Node 3 = 0.2630 


Traffic not Perceived as Threat, Node 5 (manned) 

This node represents the probability that a pilot sees a traffic aircraft which is in a collision 
course or near collision course but does not think the traffic is a threat and does not take any 
evasive action. This number is based on discussions with pilots and aeronautical experts and is 
not based on any experimentation, analysis or simulation. 

Node 5 = 1 x 10“ 2 to 1 x 10 3 
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Late or no Maneuver, Node 7 (manned) 

Probability that a pilot sees the traffic as a threat but either cannot determine an evasive 
maneuver, takes too long to determine a maneuver leaving insufficient time to execute, or 
initiates the maneuver late. In the case where a CAS is on-board, this node also reflects the 
probability that the CAS logic issues a resolution advisory (RA), but the pilot does not execute 
the advisory or executes late. (Note that for the analysis presented in this document, it is assumed 
that no CAS logic exist on-board the manned aircraft.) This number is based on discussions with 
pilots and aeronautical experts and is not based on any experimentation, analysis or simulation. 

Node 7 = 1 x 10 1 to 1 x 10 2 

Executed Incorrectly, Node 8 (manned) 

Pilot determines a maneuver which will provide avoidance, but the execution does not follow the 
intended maneuver. This could be the result of insufficient yoke force, incorrect bank angle to 
accomplish the intended turn rate, incorrect thrust setting for the maneuver, etc. This number is 
based on discussions with pilots and aeronautical experts and is not based on any 
experimentation, analysis or simulation. 

Node 8 = 1 x 10" 2 to 1 x 10" 6 


Pilot(s) Determine Resolution, Node 10 (manned) 

When there is a collision avoidance system (CAS) on-board, the pilot will have the option of 
following the resolution advisory, RA, issued by the CAS or determining his own resolution 
maneuver. It is possible that the pilot determines that a resolution maneuver is necessary but the 
CAS logic does not issue an RA. It is also possible that the CAS issues an RA but the pilot 
determines that no maneuver is needed. The calculation for the baseline in this document is made 
without CAS. Therefore, the pilot will be determining all resolutions and this node is set to 1. 

Pilot(s) see traffic, Node 14 (manned) 

Complement probability to Node 3. The Subtree B estimates the probability that given two 
aircraft are not in a collision course, the avoidance maneuver causes a collision. For this to occur, 
the traffic must have been detected. 

Traffic identified as requiring avoidance, Node 15 (manned) 

Given that separation minima will be violated and the two aircraft are in a proximity trajectory 
course, the aircraft crew or the CAS logic could identify the traffic aircraft as a collision threat 
and determine an avoidance maneuver. This number is based on discussions with pilots and 
aeronautical experts and is not based on any experimentation, analysis or simulation. 
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Node 15 = 1 x 10“ 2 to 1 x 10 5 


Pilot(s) Maneuver, Node 16 (manned) 

This node estimates the probability that the pilot maneuvers after identifying the traffic as a 
threat and determining a resolution maneuver or receiving a resolution advisory from the CAS 
logic if aircraft is equipped with CAS. This node is associated with Node 7 and is the 
complement of the probability that the pilot will not maneuver. 

Pilot(s) Determine Resolution, Node 17 (manned) 

The calculation for the baseline in this document is made without CAS. Therefore, the pilot will 
be determining all resolutions and this node is set to 1. See Node 10. 

Proximity Trajectory, Node 21 

The probability of a proximity trajectory is the probability that two aircraft in their current course 
will lose separation as defined in section 1.2. Node 21 is the probability that 2 aircraft are in a 
loss of separation course or in a collision course. For Node 21 probability, a proximity trajectory 
includes a MAC trajectory. The trajectory is considered a proximity trajectory if the two aircraft 
are less than 120 seconds to closest point of approach, CPA. That is, if no action is taken by 
either aircraft, the aircraft will experience a loss of separation or a MAC in less than 120 seconds. 
The time constraint is due to ATC automation. Typically, ATC centers have collision avoidance 
logic which will warn controllers of an impeding loss of separation 120 seconds before closest 
point of approach. Therefore, for a proximity course less than 120 seconds to CPA, it is assumed 
that ATC and procedures will not be a contributing factor to collision avoidance. 

To compare the ability of a piloted aircraft to avoid a collision with the ability of an unmanned 
aircraft to avoid a collision, the value of node 21 is set to 1. That is, procedures in the NAS and 
ATC services will be factored out for the comparison. In setting this probability to 1, there is an 
implicit assumption that a manned aircraft and an unmanned aircraft have the same probability of 
being in a proximity trajectory. This assumption is not necessarily valid because manned aircraft 
and unmanned aircraft have nominally different missions and flight profiles. To calculate the 
overall collision probability of manned and unmanned aircraft, the probability of a proximity 
trajectory will be revised as more knowledge is gained about the operations of unmanned 
aircraft. 

Pilot does not See Traffic, Node 22 (unmanned) 

Probability that a remote pilot does not scan the traffic display of the control station and does not 
see the traffic aircraft before a collision. This probability, in contrast to the manned aircraft, is 
not affected by visual acuity, size of the traffic aircraft, encounter geometry, and weather 
conditions. It is affected by work load, attentiveness, training, type of display, and other factors. 
There is very limited or no data available for remotely operated aircraft on collision trajectories. 
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In a civilian airspace environment under positive air traffic control, an aircraft will fly ten of 
thousands of hours without experiencing a proximity trajectory undetected by ATC and even 
more rarely an undetected collision trajectory. The probability assigned to this node is based on 
discussions with pilots and aeronautical experts and is not based on any experimentation, 
analysis or simulation. 

Node 22 = 1.0 x 10 2 to 1.0 x 10 4 

Late or no Maneuver, Node 24 (unmanned) 

Probability that a pilot sees the traffic as a threat but either cannot determine an evasive 
maneuver, takes too long to determine a maneuver leaving insufficient time to execute, or 
initiates the maneuver late. In the case where a CAS is at the control station, this node also 
reflects the probability that the CAS logic issues a resolution advisory (RA), but the pilot does 
not execute the advisory or executes late. Reference [1] estimates the probability of a slow 
response to a TCAS RA in the 0.5444 to 0.680 range. However, for this node, a late maneuver 
does not mean a slow reaction but rather sufficiently late that the maneuver is ineffective to avoid 
the collision. The probability assigned to this node is based on discussions with pilots and 
aeronautical experts and is not based on any experimentation, analysis or simulation. 

Node 24 = 1.0 x 10" 1 to 1 x 10" 3 

Executed Incorrectly, Node 25 (unmanned) 

Pilot determines a maneuver or receives a resolution advisory which will provide avoidance. 
However, the execution does not follow the intended maneuver. The probability is obtained from 
reference [1]. Reference [1] provides a probability that given an ACAS (TCAS) vertical 
resolution advisory, the pilot execute the resolution in the wrong direction. That is, if the 
resolution is to climb, the pilot descends and if it is to descend, the pilot climbs. 

Node 25 = 8.0 x 10 2 to 1.0 x 10 3 

Traffic not Perceived as Threat, Node 26 (unmanned) 

Probability that a pilot sees a traffic aircraft which is in a collision course or near a collision 
course but does not think the traffic is a threat. This probability does not include any type of 
warning from a CAS logic. It is the probability that by looking at a symbol on a screen, the 
apparent motion of the symbol on the screen, and the altitude and rate of change of altitude of the 
traffic, the pilot can identify the traffic as a threat. The type and design of the display will have a 
significant impact in this probability. For example, a horizontal (view from the top) display will 
make easier to detect traffic at co-altitude approaching from a given bearing. A traffic aircraft at 
closely the same horizontal location as the ownship and threatening the ownship by climbing or 
descending will be very difficult to detect as the ownship and traffic aircraft symbols will 
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overlap. The altitude must be read from the numerical value of the symbol, Figure 2.15. 



Figure 2.15. Example of Traffic Display. 

The probability assigned to this node is based on discussions with pilots and aeronautical experts 
and is not based on any experimentation, analysis or simulation. 

Node 26 = 1.0 x 10 2 to 1.0 x 10 3 

Pilot Determines Resolution, Node 31 (unmanned) 

This is the probability that the pilot determines an evasive maneuver given that there is no 
Resolution Advisory from the CAS logic or that the pilot ignores the RA and follows his own 
evasive maneuver. Whether or not a pilot decides to follow the CAS logic RA or perform his 
own avoidance maneuver depends on operational guidelines, training and other human factors. 
This node is the complement to Node 33. 


Node 31 = 1 - Node 33. 

2.5.4 Logic Events 

logic events refer to the alerts and resolution maneuvers that the CAS, the pilot or the controller 
(if a controller is involved) derive based on the data or the perception of the situation. 
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CAS Does not Warn, Node 4 (manned) 

This node is in the tree for completness and future analysis. The probability of collision for the 
baseline manned aircraft is computed without an on-board collision avoidance system (CAS). 
This node value is set to 1 . 

CAS Logic Does not Warn, Node 6 (manned) 

This node is in the tree for completeness and future analysis. The probability of collision for the 
baseline manned aircraft is computed without an on-board collision avoidance system (CAS). 
This node value is set to 1 . 


Resolution Incorrect, Node 11 (manned) 

Probability that a pilot-generated resolution maneuver fails to avoid the collision. This could be 
the result of an incorrect assessment of the encounter or that the traffic maneuvers in a way that 
negates the avoidance maneuver of the ownship. Rules of the road as defined in the Federal 
Aviation Regulations, 14 CFR Part 91.113 will lessen the chances of two aircraft maneuvering 
into each other. The probability assigned to this node is based on discussions with pilots and 
aeronautical experts and is not based on any experimentation, analysis or simulation. 

Node 11 = 1.0 x 10- 1 to 1.0 x 10 3 

CAS Logic Issues RA/ Pilot Selects, Node 12 (manned) 

This node is in the tree for completeness and future analysis. The probability of collision for the 
baseline manned aircraft is computed without an on-board collision avoidance system. This node 
value is set to 0. 

RA Incorrect, Node 13 

Probability that if there is a collision avoidance system on-board, the resolution generated by the 
CAS logic fails to solve the collision. This node is in the tree for completeness and future 
analysis. The probability of collision for the baseline manned aircraft is computed without an on- 
board collision avoidance system. This node value is set to 0. 

Resolution is MAC Trajectory, Node 18 (manned) 

Probability that the resolution maneuver determined by the pilot reduces the separation at the 
closest point of approach causing a collision where the original trajectories were not collision 
trajectories. This probability is dependent on the size of the proximity trajectory. The larger the 
proximity trajectory, the lower the probability that an aircraft will maneuver into the traffic. The 
probability for this node is estimated for the proximity trajectory as defined in Section 1.5. 
Assuming a purely random maneuver, a geometric analysis similar to that for Nodes 1 and 2 can 
be performed. This yields a probability in the order of 0.005 of collision. If the pilots follow the 
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rules of the rode, the probability of maneuvering towards the traffic should be significantly 
reduced. 

Node 18 = 5.0 x 10 3 to 1.0 x 10 5 

CAS Logic Issues RA/Pilot Selects, Node 19 (manned) 

This node is in the tree for completeness and future analysis. The probability of collision for the 
baseline manned aircraft is computed without an on-board collision avoidance system. This node 
value is set to 0. 

RA is MAC trajectory, Node 20 (manned) 

Probability that resolution advisory maneuver reduces the separation at the closest point of 
approach causing a collision where the original trajectories were not collision trajectories. This 
node is set to 0 based on the assumption that CAS is not present. 

CAS Logic Does not Warn of Traffic Presence, Node 23 (unmanned) 

If the unmanned aircraft's control station is equipped with collision avoidance system, this is the 
probability that the CAS does not alert the pilot of the presence of traffic. This probability is 
highly dependent on the system design. It could be possible that it is undesirable to have a traffic 
presence warning as this might cause excessive alerts which lead to pilots disregarding warnings. 
A traffic presence warning could also be user selectable and/or sensitivity adjustable with regard 
to horizontal and vertical distance. In higher density conditions, sensitivity could be reduced to 
avoid excessive alerts. The probability assigned to this node is based on discussions with system 
designers, pilots and aeronautical experts and is not based on any experimentation, analysis or 
simulation. 

Node 23 = 1.0 to 1.0 x 10' 6 

CAS Logic Does not Warn of Collision Trajectory, Node 27 (unmanned) 

If the unmanned aircraft's control station is equipped with collision avoidance system, this is the 
probability that based on the data supplied to the CAS logic, the logic does not warn the pilot of 
an impending collision. This probability is based on simulations of conflict and collision 
avoidance logic. The distinction between conflict and collision avoidance is determined by the 
time length of the look ahead time and the horizontal and vertical separation objectives. Conflict 
avoidance involves look ahead times in the order of 5 minutes to several hours and horizontal 
and vertical separation of 5 NM and 1000 feet, respectively. Collision avoidance deals with look 
ahead times in the order of 20 seconds to a few minutes and horizontal and vertical separations of 
less than 0.5 NM and 500 feet respectively. Reference [10] shows that, given accurate 
surveillance data, the conflict detection logic was successful in detecting all conflicts in more 

than 3.3 x 10 5 encounters. Reference [11] shows results of conflict and collision avoidance 
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experiments for a terminal or merging area environment in which all conflicts/collision trajectory 
were detected for more than 300 operations. The case for erroneous or non-existent data is not 
considered by this node and is included in node 28. Based on the simulation experiments, a 
probability is assigned for CAS logic not warning of collision trajectory. 


Node 27 = 3.03 x 10 6 


Resolution Incorrect, Node 32 (unmanned) 

Probability that a pilot-generated resolution maneuver fails to avoid the collision. The maneuver 
is generated by observing the control station traffic display and determining a suitable maneuver 
to avoid the collision. The probability assigned to this node is based on discussions with pilots 
and aeronautical experts and is not based on any experimentation, analysis or simulation. 

Node 32 = 0.2 to 0.1 

CAS Logic Issues RA, Node 33 (unmanned) 

Probability that the CAS logic issues an RA and that the pilot decides to follow the CAS logic 
RA and does not follow his own resolution maneuver. The CAS logic probability follows from 
the same simulation experiments as Node 27. Whether or not a pilot decides to follow the CAS 
logic RA or perform another avoidance maneuver depends on operational guidelines, training 
and other human factors. The probability that the logic issues an RA is expected to be greater 

than 3.3 x 10 5 . Of 170 TCAS reported incidents in reference [13], 8% of pilots chose to ignore 
the RA advisory and maneuver based on their own judgment. However, this data is for manned 
aircraft with visual capabilities. It is expected that a remote pilot using a traffic display will be 
more incline to follow the RA. The probability that the pilot follows the logic is the dominant 
factor for this node. 


Node 33 = 0.92 to 0.999 


RA incorrect, Node 34 (unmanned) 

Probability that the resolution maneuver issued by the CAS logic fails to avoid the collision. In 
collision avoidance logic, the resolution calculation is based on projecting the state and velocity 
vectors of the intruder aircraft and determining a change in velocity vector of the own ship to 
achieve the required avoidance. When the traffic aircraft alters its course, the resolution 
generated by the ownship to avoid the collision might no longer achieve its objective. This is the 
main reason why an RA fails to avoid the collision. Other less influential factors that might make 
the RA fail include aircraft performance, proximity to terrain, high density traffic environment, 
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surveillance data error, etc. From reference [1], the probability that a TCAS generated RA fails to 

resolve a collision is estimated to be 9.209 x 10~ 2 to 8.873 x 10" 2 and the probability used for this 
node. 


Node 34 = 9.209 x 10' 2 to 8.873 x 10' 2 


Pilot detects traffic, Node 35 (unmanned) 

This node is the complement to traffic not detected. Traffic is not detected if not on display or the 
remote pilot fails to see it on the display. 


Traffic identified as requiring avoidance, Node 36 (unmanned) 

Given that separation minima will be lost and the two aircraft are in a proximity trajectory 
course, the aircraft crew or the CAS logic could identified the traffic aircraft as a collision threat 
and determine an avoidance maneuver. This number is based on discussions with pilots and 
aeronautical experts and is not based on any experimentation, analysis or simulation. 

Node 15 = 1 x 10“ 2 to 1 x 10 5 

Pilot(s) Maneuver, Node 37 (unmanned) 

This node estimates the probability that the pilot maneuvers after identifying the traffic as a 
threat and determining a resolution maneuver or receiving a resolution advisory from the CAS 
logic. This node is associated with Node 24 and is the complement of the probability that the 
pilot will not maneuver. 

Mode 37 = 1 - Node 24 

Pilot Determines Resolution, Node 38 (unmanned) 

This node is the complement to Node 40. 

Node 38 = 1- Node 40 

Pilot resolution is MAC Trajectory, Node 39 (unmanned) 

Probability that the resolution maneuver determined by the pilot reduces the separation at the 
closest point of approach causing a collision where the original trajectories where not collision 
trajectories. This probability is dependent on the size of the proximity trajectory. The larger the 
proximity trajectory, the lower the probability that an aircraft will maneuver into the traffic. The 
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probability for this node is estimated for the proximity trajectory as defined in Section 1.5. 
Assuming a purely random maneuver, a geometric analysis similar to that for Nodes 1 and 2 can 
be performed. This yields a probability in the order of 0.005 of collision. If the pilots follow the 
rules of the rode, the probability of maneuvering towards the traffic should be significantly 
reduced. 

Node 18 = 5.0 x 10 3 to 1.0 x 10‘ 5 

CAS Logic Issues RA, Node 40 (unmanned) 

Probability that the CAS logic issues and RA and that the pilot decides to follow the CAS logic 
RA and does not follow his own resolution maneuver. The probability that the CAS issues an RA 
is lower than for Node 33 because the aircraft are in a proximity trajectory and not in a collision 
trajectory. Whether or not the logic issues the RA depends on the sensitivity of the logic and the 
trade off between false alerts and untimely alerts. The dominant factor in this probability, 
however, is the pilots willingness to follow the RA. This probability is estimated from [13]. 


Node 33 = 0.92 to 0.999 


RA is MAC trajectory, Node 41 (unmanned) 

Probability that the RA issued by the CAS logic will induce a collision when the aircraft are not 
in a collision course. This probability is estimated from geometrical considerations and from 
reference [1], 

Node 41 = 5.68 x 10" 4 to 1 x 10" 5 


3. Encounter Geometries 

The encounter geometries presented in this section are the encounters used in the TCAS II safety 
analysis [4], The encounters are divided into 19 classes. These encounters were obtained from 
terminal area radar observations in the mid 1980s. The subset of encounters used were 
encounters considered to have generated an RA by a TCAS unit. Each encounter class is given a 
probability of occurrence associated with the frequency of observation. The encounter 
classification is based on the vertical speed of the aircraft before and after the closest point of 
approach, CPA. For example, given aircraft A and B in the encounter, Class 1 defines the 
encounter in which aircraft A is (changing altitude) before the CPA, aircraft A is changing 
altitude after the CPA, and aircraft B is in level flight before and after the CPA. 

In this report, the Classes 1, 3, and 14 are subdivided into Classes la, lb, 3a, 3b, 14a, and 14b. 
The subdivision of the encounters into a and b makes a distinction on whether changing altitude 
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is a climb or a descent. The distinction is made because it is expected that the ownship will have 
different visual acquisition rates for an intruder that is climbing as opposed to an intruder that is 
descending. 

The encounter geometry data does not contain crossing angle information. The crossing angle is 
the angle formed by the aircraft's paths as observed from above. That is, the crossing angle is the 
difference in bearing of the two aircraft. See Figure 3.23. For the calculation of visual acquisition 
probabilities, a uniform distribution is assumed for the crossing angle. 

The data used for the encounter geometries is based on terminal area radar observation of 
manned aircraft. These data might not be representative of en-route encounters and unmanned 
aircraft flight profiles. As more knowledge is gained about unmanned aircraft operations, these 
data will be revised. 

The encounter classes are represented in the vertical plane in Figures 3.1 to 3.22. 





Figure 3.1. Class la 


Figure 3.2. Class 1b 



CPA 




PCA 


Figure 3.3. Class 2. 


Figure 3.4. Class 3a. 
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Figure 3.5. Class 3b. 


Figure 3.6. Class 4. 




Figure 3.9, Class 7. 


Figure 3.10. Class 8. 



Figure 3.11. Class 9. 


Figure 3.12. Class 10. 
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Figure 3.13. Class 11. 




CPA 


Figure 3.14. Class 12. 
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Figure 3.15. Class 13. Figure 3.16. Class 14a. 



Figure 3.17. Class 14b. Figure 3.18. Class 15. 
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Figure 3.19. Class 16. 


Figure 3.20. Class 17. 
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Figure 3.21. Class 18. Figure 3.22. Class 19. 


The encounter classes are not all likely to occur with the same frequency. The probability of 
occurrence given to each of the encounter class in reference [4] is shown in Table 3.1. 


Class 

P Occurrence 

Class 

P Occurrence 

Class 

P Occurrence 

1 

0.01733 

7 

0.00002 

13 

0.07914 

2 

0.00092 

8 

0.00002 

14 

0.05960 

3 

0.00046 

9 

0.00002 

15 

0.01406 

4 

0.01205 

10 

0.17004 

16 

0.04914 

5 

0.00046 

11 

0.57531 

17 

0.00008 

6 

0.00042 

12 

0.01574 

18 

0.00510 


19 

0.00008 


Table 3.1. Probability of Occurrence. 

For each of the class encounter geometry, the encounter crossing angle is varied from 0 to 180 
degrees in 5 degrees increments. The top view of the encounters is show in Figure 3.23. 
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Figure 3.23. Top view of Encounter Geometries Showing Crossing Angle. 


The probabilities for the tree nodes are computed for 37 crossing angles for each encounter class. 
The average probability of collision for the 37 angles is computed and used for the class node on 
the event tree. 

Descend and climb rates are set to 500 feet per minute for constant climbs or descents. Descend 
and climb rates are set to 1000 feet per minute when the aircraft goes from level flight to altitude 
change or from altitude change to level flight. For encounter classes 3a, 3b and 9, the aircraft are 
climbing or descending to a point 20 seconds before CPA and leveling off for the segment 20 to 
Tyisuai seconds before CPA. 


4. Visual 3D Visual Acquisition Program 

The visual 3D program was developed to calculate the probability that a pilot or pilots can see a 
traffic aircraft during encounters in which the vertical velocity of the intruder or ownship might 
not be zero. That is, one or both aircraft might be climbing or descending. This program is a 
three dimensional extension of the two dimensional SEE1 program reported in [2], It is written 
in Java and can be executed in various computer platforms. The program calculates the 
probability that a pilot or pilots will be able to visually detect a target aircraft t seconds before the 
CPA. The program takes 20 inputs and outputs the acquisition probability. The inputs to the 
program are: 

Type of aircraft, traffic True air speed, own True air speed, traffic 
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Crossing angle 
Climb rate, own 
Climb rate, traffic 
Visual range 
Resolution limit 
Field of view, H up 


Field of view, H down 

Field of view, left 

Filed of view, right 

Search, pilot 1 

Search, pilot 2 

Beta 0, search effectiveness 


Beta 1, search effectiveness 
Tau 1, beta transition, time 
Tau 2, Pacq, time 
Dl, beta transition, range 
D2, Pacq, range 


Default values for these parameters are: 

Visual range: 20 nautical miles 
Resolution limit: 1 arc-min 
Field of view, H up: 60 degrees 
Field of view, H down: -75 degrees 
Field of view, left: -120 degrees 
Field of view, right 90 degrees 


Search pilot 1 : 1 (pilot present) 
Search pilot 2: 1 (pilot present) 
Beta 0: 17000 steradian-sec 
Beta 1: 17000 steradian-sec 
Tau 1: 180 seconds 
Tau 2: 12 seconds 


Figure 4.1 shows a 3-dimensional representation of an encounter with the parameters used by the 
Visual 3D program. 
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Figure 4.1. Three Dimensional Encounter as Constructed by the Visual 3D 

Program. 

These parameters are derived from the inputs given to the program and include rate of change of 
aircraft range and angles of view. 

The program calculates the probability of visual acquisition for linear trajectories. Trajectories 
that change vertical speeds and/or are not linear horizontally are approximated by segments of 
linear trajectories. The program does not provide for night time visual acquisition calculations. 

5. Summary and Conclusion 

The event trees presented in this paper are a framework to calculate the probability of collision 
for manned aircraft and for unmanned aircraft. The top level tree, Figure 2.3, is common to both 
trees. Subtrees A and B have nodes that are more specific to manned and unmanned flight 
operations. The trees can be used to calculate an absolute probability as well as a risk ratio. A 
risk ratio calculation or relative probability has two main advantages: 


41 



1. By changing only parts of an event tree for the comparison, errors in estimating basic 
events affecting both trees are factored out. The effort of the analysis can be focused on 
the differences between the trees, which are the more critical factors. 

2. Comparing the tree of interest to a well understood and established tree add credibility 
and accuracy to the calculation. When a comparison is made between a unmanned 
collision tree and a manned collision tree, the manned collision tree can be checked 
against actual data to make sure that the tree’s absolute value is accurate. Historical data 
of collision between manned aircraft can be obtained from the National Transportation 
Safety Board. This process will be similar to calibrating the tree that is used as the 
reference. However, the collision between aircraft flying in Class A airspace (FL180 and 
above) is extremely rare and presents some statistical challenges. 

Using the values assigned to the nodes in Section 2.5, the following risk ratio table is calculated. 
The Risk Ratio is the probability of collision of a unmanned aerial system divided by the 
probability of collision of a manned aircraft using only see and avoid under VMC (Visual 
Meteorological Conditions) with no collision avoidance system on-board. The upper bound is the 
probability calculated using the numbers in the range (Section 2.5) that gives the worst outcome. 
The lower bound is calculated using the numbers is Section 2.5 for the best collision probability. 


Risk Ratio 

See and Avoid, Manned 

Probability of collision 
Upper bound 

Probability of collision 
Lower bound 

Sense and Avoid 
UAS (Unmanned) 

Probability of collision 
Upper bound 

0.5846 

0.9666 

Probability of collision 
Lower bound 

0.1938 

0.3203 


Table 5.1. Risk Ratio of Sense and Avoid vs. See and Avoid. 

When calculating the probability of collision for a manned aircraft, the dominant factor is the 
visual acquisition probability. In a clear day with 20 NM visibility and the other factors 
described in Section 2.5.3, a pilot has approximately a 25% probability of not seen a traffic 
aircraft in a collision course. 

A remotely piloted unmanned aircraft has a much higher probability of detecting a traffic aircraft 
in a collision course. This was based on Mode-S surveillance or ADS-B following RTCA-DO- 
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260 or 260A requirements. The dominant factor affecting the probability of collision for a 
remotely piloted UAS is implementing the avoidance maneuver. 

The risk ratio calculations suggest that an UAS can achieve a comparable level of safety to see 
and avoid. It is important to note, however, that some of the values assigned to the nodes are 
based on opinions from pilots and aeronautical experts. More extensive analysis and simulation 
is required to validate the node probabilities. Also noteworthy is that these probabilities are for 
the ownship avoiding the traffic. It does not take into account the probability that the traffic 
aircraft will see/sense the ownship and avoids it in cases that the ownship fails to see/sense the 
traffic. 

The risk ratio calculations have been made for a manned aircraft with two pilots and without 
collision avoidance equipment on-board. The event trees and analysis structure, however, are 
built such that the calculations can be made for a maned aircraft with a collision avoidance 
system on-board such as TCAS. These calculations could be performed in future work. 


6. Acronyms 

ACAS - Airborne Collision Avoidance System 

CAS - Collision Avoidance System 

CCA - Cooperative Collision Avoidance 

Cl - Controller Instruction 

CPA - Closest Point of Approach 

ELOS - Equivalent Level of Safety 

FL - Flight Level; FL400 = 40,000 feet 

MAC - Mid-Air Collision 

NAS - National Airspace System 

NASA - National Aeronautic and Space Administration 

NMAC - Near Mid-Air Collision 

RA - Resolution Advisory 

ROA - Remotely Operated Aircraft 

RVSM - Reduced Vertical Separation Minima 

TCAS - Traffic and Collision Avoidance System 

UAS - Unmanned Aircraft System 
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